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Abstract 

This  paper  presents  a  patient  safety  applicator  tool  for  implementing  and 
assessing  patient  safety  systems  in  health  care  institutions.  The  applicator  tool 
consists  of  critical  processes  and  performance  measures  identified  in  the  context 
of  the  2003  Malcolm  Baldrige  National  Quality  Award  (MBNQA)  Health  Care 
Criteria  for  Performance  Excellence.  The  Delphi  technique  was  used  for  gaining 
consensus  from  a  group  of  experts  and  forecasting  significant  issues  in  the  field  of 
the  Delphi  panel’s  expertise.  Data  collection  included  a  series  of  questionnaires 
where  the  first-round  questionnaire  was  based  on  literature  review  and  the 
MBNQA  criteria  for  excellence  in  health  care.  Data  were  tested  by  an  instrument 
review  panel  of  experts.  Twenty-three  experts  (MBNQA  health  care  reviewers 
and  senior  health  care  administrators  from  quality  award  winning  institutions) 
representing  18  States  participated  in  the  survey  rounds.  The  study  addressed 
three  research  questions: 

1 .  What  critical  processes  should  be  included  in  health  care  patient  safety 
systems? 

2.  What  perfonnance  measures  can  serve  as  indicators  of  quality  for  the 
processes  critical  for  ensuring  patient  safety? 

3.  What  processes  will  be  critical  for  patient  safety  in  the  future? 

This  study  is  significant  because  the  results  are  expected  to  assist  health  care 
institutions  seeking  to  develop  high  quality  patient  safety  programs,  processes, 
and  services.  The  identified  critical  processes  and  performance  measures,  which 
extend  the  Malcolm  Baldrige  established  framework  into  the  area  of  patient  safety 
and  which  are  presented  as  a  three-level  applicator  tool,  can  serve  as  a  means  of 
evaluating  existing  patient  safety  initiatives  and  of  guiding  the  strategic  planning 
of  new  safety  processes.  The  patient  safety  applicator  tool  utilizes  a  systems 
approach  and  will  support  health  care  senior  administrators  in  achieving  and 
sustaining  improvement  results.  It  also  will  also  assist  health  care  institutions  in 
using  the  MBNQA  Health  Care  Criteria  for  Performance  Excellence  for  self- 
assessment  and  quality  improvement. 


Introduction 

Strong  U.S.  Government  leadership  is  essential  to  support  the  coordination 
and  institutionalization  of  health  care  quality  priorities  and  in  implementation  of 
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clinical  performance  measures.  The  Government  has  been  asked  to  provide 
guidance  on  development  of  standards  of  care  and  patient  safety  accountability 
throughout  the  health  care  system  in  order  to  assist  in  the  establishment  of  the 
following:  performance-based  payment  policies;  removal  of  major  financial 
barriers  to  quality  improvement;  and  investment  in  quality  and  safety 
infrastructure,  research,  and  training.1  Although  patient  safety  approaches  have 
been  targeted  by  recent  research  and  requirements  of  accrediting  organizations, 
deciding  which  approach  is  best  remains  controversial.  Current  patient  safety 
regulations  have  been  implemented  mainly  in  response  to  the  number  of  observed 
patient  safety  issues  reported  in  professional  publications.  If  a  problem  has  not 
been  recorded  or  reported,  which  is  often  the  case  with  patient  safety-related 
issues,  it  is  most  likely  not  reflected  in  patient  safety  requirements.  No 
systemwide  approach  or  model  has  been  agreed  upon  thus  far.  Consensus  building 
methods  that  use  experts  as  a  source  of  knowledge  and  information,  coupled  with 
a  comprehensive  literature  review,  are  reliable  approaches  in  patient  safety 
research,  a  field  in  particular  need  of  rigorous  qualitative  and  consumer-oriented 
research  to  fill  an  existing  methodological  gap.2  The  output  from  consensus 
approaches,  such  as  Delphi,  is  not  an  end  in  itself;  rather,  dissemination  and 
implementation  of  the  study  findings  are  the  ultimate  aims  of  consensus  activities 
and  are  intended  to  guide  health  policy  decisionmaking,  clinical  practice,  and 
research.  This  study  is  based  on  the  Malcolm  Baldrige  National  Quality  Award 
(MBNQA)  model  that  has  been  refined  over  the  last  17  years  and  adopted  in  a 
number  of  health  care  institutions  for  improvement  of  overall  organizational  and 
clinical  outcomes.  This  study  strives  to  expand  in  theoretically  meaningful  and 
practically  applicable  ways  the  existing  knowledge  in  the  area  of  patient  safety.  It 
provides  a  three-level  applicator  tool  intended  to  assist  health  care  leadership  in 
organizations  that  are  at  different  stages  of  their  patient  safety  and  quality 
improvement  j  ourney . 


Methodology 

Malcolm  Baldrige  quality  framework 

The  methodological  model  for  this  study  was  the  MBNQA  framework  for 
performance  excellence  in  health  care.  The  MBNQA  was  established  in  1987  to 
address  the  importance  of  quality  as  the  most  significant  factor  for  the  trade 
balance  of  the  United  States  and  is  the  most  prestigious  national  quality  award  in 
the  United  States.  The  award  is  performance  oriented  and  is  clearly  focused  on  the 
importance  of  integration  of  quality  in  institutional  business  planning.4  Judges  and 
examiners  for  the  MBNQA  are  selected  on  the  basis  of  their  experience,  quality 
expertise,  and  peer  recognition.  Presidential  involvement  in  the  award  process 
(the  award  is  presented  by  the  President  of  the  United  States  or  the  Secretary  of 
Commerce)  ensures  national  impact,  visibility  and  prestige  for  the  award  winners, 
wide  peer  recognition  for  winners’  achievements,  and  deployment  of  the  winners’ 
results  throughout  industries  in  the  Nation.5  The  health  care  criteria  for 
performance  excellence  were  introduced  in  January  1998.  The  MBNQA 
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framework  consists  of  1 1  core  values  and  concepts,  embodied  in  seven  categories: 
(1)  leadership;  (2)  strategic  planning;  (3)  focus  on  patients,  other  customers,  and 
markets;  (4)  measurement,  analysis,  and  knowledge  management;  (5)  staff  focus; 
(6)  process  management;  and  (7)  organizational  perfonnance  results.6 

Thus  far,  there  have  been  four  award  winners  in  the  category  of  health  care: 
SSM  Health  Care  (2002),  Saint  Luke’s  Hospital  of  Kansas  City  (2003),  Baptist 
Hospital,  Inc.  (2003),  and  Robert  Wood  Johnson  University  Hospital  Hamilton 
(2004).  All  award  winners  collect  data,  allowing  for  tracking  of  overall 
performance  and  identification  of  opportunities  for  improvement  within  the 
respective  institution.  They  have  made  an  effort  to  align  institutional  operations 
from  top  to  bottom  utilizing  a  variety  of  performance  improvement  approaches. 

Delphi  methodology 

In  this  study,  the  Delphi  method  was  used  to  reach  consensus  among  health 
care  quality  improvement  and  patient  safety  experts.  The  Delphi  method,  a  useful 
way  of  identifying  and  measuring  uncertainty,  has  been  widely  utilized  in  medical 
and  health  services  research  to  define  professional  roles  and  to  clarify  issues  in 
health  service  organizations.  It  also  has  been  used  to  aid  in  the  design  of 
educational  programs,  make  long-tenn  projections  of  need  for  care  for  particular 
population  groups,  develop  criteria  for  appropriateness  of  interventions  and  clinical 
protocols,  and  define  adverse  effects  of  reducing  medical  staffing  levels.3, 7  14  The 
Delphi  method  gives  panel  members  an  avenue  for  asynchronous  interaction, 
where  they  choose  to  participate  in  the  group  communication  process  at  their 
convenience  and  relate  to  questions  they  feel  best  qualified  to  discuss.  The  Delphi 
method  is  a  collaborative  expert  system7  where  the  experts  are  provided  with  a 
Delphi  design  and  where  they  dynamically  and  actively  contribute  their 
knowledge  to  the  system.  This  method  significantly  broadens  knowledge  and 
effective  decisionmaking  in  health  and  social  care. '  "  The  selection  of  criteria  that 
would  qualify  an  individual  to  participate  on  a  Delphi  panel  depends  on  the  aims 
and  context  of  the  particular  study.7, 16 

Study  data  collection  included  a  series  of  questionnaires.  The  first-round 
questionnaire  was  based  on  a  literature  review  and  the  MBNQA  criteria  for 
excellence  in  health  care.  Subsequently,  23  health  care  quality  improvement 
experts  (MBNQA  health  care  reviewers  and  senior  health  care  administrators 
from  quality  award  winning  institutions)  representing  18  States  participated  in  the 
survey  rounds  (Table  1). 

The  study  addressed  three  research  questions: 

1 .  What  critical  processes  should  be  included  in  health  care  patient  safety 
systems? 

2.  What  perfonnance  measures  can  serve  as  indicators  of  quality  for  the 
processes  critical  for  ensuring  patient  safety? 

3 .  What  processes  are  critical  for  patient  safety  in  the  future? 
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Table  1.  Delphi  expert  panel  participants 


Participant  characteristics 

Number  of 
participants 
(N  =  23) 

Examiners  for  the  MBNQA  (including  ranks  of  examiner,  senior,  and  judge) 

14 

M.D.  and/or  Ph.D. 

6 

Registered  nurse  or  nurse  practitioner 

7 

Representative  of  an  institution  that  has  applied  for  or  won  the  MBNQA  in 
the  health  care  category 

6 

Representative  of  a  health  care  institution  that  has  won  a  State  quality 
award 

6 

Senior  hospital  administrator 

20 

Leader  in  a  national  organization  for  health  care  quality  and  patient  safety 

1 

Leader  in  a  State  organization  for  health  care  quality  and  patient  safety 

1 

Female 

17 

Male 

6 

The  study  had  two  major  phases:  (1)  creation  of  the  original  survey  instrument 
and  initial  establishment  of  its  content  validity  by  a  review  panel  of  three  quality 
improvement  and  patient  safety  experts,  and  (2)  conducting  three  iterations  of  the 
survey  with  the  health  care  quality  improvement  experts  on  the  Delphi  panel. 

The  original  survey  instrument  consisted  of  31  critical  processes  and  58 
performance  measures  structured  along  the  framework  of  the  MBNQA  Health 
Care  Criteria  for  Performance  Excellence.  Each  performance  measure  was  linked 
to  a  critical  process,  and  each  critical  process  had  one  or  more  associated 
performance  measures.  The  critical  processes  and  performance  measures  included 
in  the  initial  instrument  were  identified  through  an  extensive  literature  review  of 
professional  publications  on  patient  safety  and  health  care  quality  improvement. 
During  the  first  survey  iteration,  the  Delphi  experts  assessed  the  importance  of 
patient  safety  critical  processes  and  performance  measures  included  in  the  survey 
instrument.  The  Delphi  experts  were  asked  to  rank  the  critical  processes  and 
performance  measures  on  a  Likert-type  scale  indicating  a  degree  of  importance 
from  “very  important”  (rank  of  4)  to  “unimportant”  (rank  of  1).  Panelists  were 
also  asked  to  add  any  critical  process  or  performance  measure  they  believed 
should  be  included  in  the  questionnaire  and  were  given  the  opportunity  to  edit  all 
processes  and  measures  as  they  deemed  appropriate. 

The  second-round  questionnaire  included  all  critical  processes  and 
performance  measures  in  the  original  survey  instrument  along  with  additional 
critical  processes  and  performance  measures  suggested  by  panelists  in  the  first 
round.  Three  critical  processes  and  six  performance  measures  were  corrected,  and 
eight  new  critical  processes  and  12  performance  measures  were  added.  For  each 
critical  process  and  performance  measure,  the  group  rank  mean  from  the  first 
round  was  included,  along  with  the  individual  participant  rank  for  each  variable. 
Each  expert  was  then  asked  to  review  the  group  scores  and  re-evaluate  their 
individual  original  responses.  Changes  of  ranks  were  permitted  in  the  process  of 
building  consensus. 
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All  critical  processes  and  perfonnance  measures  for  which  consensus  was  not 
reached  during  the  second  survey  round  were  included  in  the  third  survey 
iteration.  All  critical  processes  and  perfonnance  measures  suggested  by  the  study 
participants  during  the  first  survey  iteration  were  also  included  in  the  third 
questionnaire.  At  the  conclusion  of  the  third  survey  round,  consensus  was  reached 
about  the  current  and  future  importance  of  patient  safety  critical  processes  and 
performance  measures.  At  the  end  of  the  study,  individual  experts  were  asked  to 
give  pennission  for  their  names  to  be  cited  as  contributors  to  this  study. 


Results 

Question  1:  What  critical  processes  should  be  included  in  health  care  patient 
safety  systems?  All  identified  patient  safety  critical  processes  were  perceived  to 
be  at  least  “important”  to  patient  safety  systems  in  health  care  institutions.  The 
Delphi  panel  perceived  the  following  as  milestones  for  implementation  of  patient 
safety  systems  in  health  care  institutions: 

•  Health  care  senior  leadership  direction. 

•  Institutional  governance. 

•  Setting  of  institutional  goals  for  perfonnance  review. 

•  Ensuring  that  health  care  information  technology  is  reliable,  secure, 
and  user-friendly. 

Question  2:  What  performance  measures  can  serve  as  indicators  of  quality  for 
the  processes  critical  for  ensuring  patient  safety?  All  performance  measures  were 
considered  either  “very  important”  or  “important”  for  implementation  of  patient 
safety  systems  in  health  care  institutions.  Thirty-one  of  the  perfonnance  measures 
in  the  patient  safety  applicator  tool  received  a  rank  of  “very  important”  to  patient 
safety  systems  in  the  present.  These  3 1  “very  important”  perfonnance  measures 
were  distributed  throughout  the  framework  categories  with  the  exception  of 
category  3.  As  reflected  by  the  group  rank  means  for  perfonnance  measures  in 
category  7,  performance  measures  based  on  accreditation  standards  are  considered 
important  for  implementation  of  patient  safety  systems  in  health  care  institutions. 

Question  3:  What  processes  are  critical  for  patient  safety  in  the  future?  The 
Delphi  experts  forecasted  that  all  critical  processes  identified  in  the  patient  safety 
applicator  tool  will  have  higher  importance  in  the  future  than  in  the  present.  All 
patient  safety  critical  processes  had  an  increased  consensus  group  rank  mean  for 
their  future  importance  in  comparison  with  the  group  rank  mean  for  their  current 
importance.  Three  of  the  patient  safety  critical  processes  received  a  perfect  score 
of  4.0  regarding  their  future  importance  to  implementation  of  patient  safety 
systems  in  the  future.  These  processes  were: 

•  Critical  process  1.1. 1 — How  senior  leaders  communicate  the  priority 
of  patient  safety  to  all  stakeholders. 
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•  Critical  process  4. 1 .2 — How  the  institution  collects,  tracks,  and 
analyzes  patient  safety  data. 

•  Critical  process  4.2.1 — How  the  institution  ensures  that  its  clinical 
information  technology  is  reliable,  secure,  and  user-friendly. 

These  processes  are  considered  essential  for  patient  safety  systems  in  health  care 
institutions. 

The  Delphi  panel  forecasted  that  patient  safety  strategy  development  and 
deployment  would  be  critical  for  the  process  of  implementation  of  patient  safety 
systems  in  the  future.  While  customer  relationship  building  was  also  perceived  to 
be  an  important  area  to  address,  obtaining  infonnation  and  feedback  from  patients 
on  patient  safety  issues  in  order  to  improve  health  care  delivery  was  given  a 
higher  priority.  The  Delphi  panel  reached  consensus  that  the  way  health  care 
institutions  design  and  detennine  process  requirements  for  their  patient  safety 
systems,  along  with  departmental  patient  safety  infrastructure  and 
interdepartmental  coordination  of  patient  safety  activities,  would  be  crucial  for 
introducing  and  improving  patient  safety.  Two  performance  measures  received  a 
perfect  score  of  4.0.  These  two  performance  measures  related  to  ensuring  a 
nonpunitive  approach  for  reporting  all  adverse  events  and  near  misses,  and 
ensuring  that  an  accessible,  confidential,  and  adequately  functioning  reporting 
system  is  in  place  for  reporting  all  adverse  events  and  near  misses. 

Inclusion  criteria  for  final  tool 

This  study  created  a  patient  safety  applicator  tool  for  an  institution-wide 
systems  approach  to  introducing,  maintaining,  and  improving  health  care  patient 
safety  systems.  The  tool  is  divided  into  three  levels:  beginner,  intermediate,  and 
advanced.  The  criteria  for  inclusion  of  critical  processes  and  performance 
measures  in  the  tool  were  based  on  the  group  mean  rank  and  the  standard 
deviation  of  the  importance  of  the  items  in  the  Delphi  surveys.  Critical  processes 
and  performance  measures  considered  “important”  (with  a  consensus  group  mean 
between  2.5  and  3.4)  or  “very  important”  (with  a  consensus  group  mean  equal  to 
or  higher  than  3.5)  for  patient  safety  systems  in  health  care  institutions  were 
included  in  the  final  patient  safety  applicator  tool.  Additionally,  each  Malcolm 
Baldrige  category  was  presented  with  its  respective  items,  areas  to  address, 
critical  processes,  and  perfonnance  measures.  Only  critical  processes  and 
performance  measures  identified  as  most  important  (highest  ranking)  per  each 
category  item  are  included  at  the  beginner  level  (Table  2). 

The  intermediate  level  includes  processes  and  performance  measures  at  the 
beginner  level  plus  essential  processes  (i.e.,  identified  as  “very  important”)  for  the 
systems  approach  for  patient  safety  systems  management  at  the  organizational 
level.  The  advanced  level  includes  the  full  patient  safety  applicator  tool  as 
identified  by  the  study  experts.  The  three  levels  of  applicator  tool  are  available  via 
the  Internet  (http://rchitexas.org/presentations/akins)  or  from  the  corresponding 
author. 
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Table  2.  Patient  safety  applicator  tool:  beginners 


Area  to  address:  Critical 
Process 


Performance  Measure 


Category  1:  Leadership 


1.1.  Institutional 
Leadership 


1.2.  Social 
Responsibility 


Senior  Leadership  Direction: 
How  senior  leaders 
communicate  the  priority  of 
patient  safety  to  ail 
stakeholders. 


Ethical  Behavior:  How  the 
institution  ensures  ethical 
communication  with 
stakeholders  in  regard  to 
patient  safety  issues. 


Functioning  institutional  systems  for 
communicating  patient  safety  policies, 
issues,  and  activities  to  all 
stakeholders;  actively  seeking 
feedback  and  use  of  the  information  for 
improvement  and  creating  a  culture  of 
safety. 

Ongoing  monitoring  of  quality  issues 
and  appropriate  procedures  are  in 
place  for  reporting  and  analysis  of 
adverse  events  and  improvement  of 
institution’s  patient  safety  systems. 


Category  2:  Strategic  Planning 


2.1.  Strategy 
Development 


2.2.  Strategy 
Deployment 


Strategic  Objectives:  How 
patient  safety  practices  are 
identified  and  translated  to 
institution’s  goals. 

Action  Plan  Development  and 
Deployment:  How  the 
institution  develops,  monitors, 
and  improves  action  plans  to 
ensure  patient  safety. 


Data  from  national  databanks  and 
practice  guidelines  from  professional 
organizations  are  incorporated  in 
institution’s  patient  safety  goals,  plans, 
and  patient  care  practices. 

Institutional  and  unit  patient  safety 
action  plans  and  systems  for  sustaining 
achieved  improvements  are  in  place 
and  are  revised  and  improved  on  a 
regular  basis. 


Category  3:  Focus  on  Patients,  Other  Customers,  and  Markets 


3.1.  Patient,  Other 
Customer,  and 
Healthcare  Market 
Knowledge 


3.2.  Patient  and  Other 
Customer  Relationships 
and  Satisfaction 


Patient  Safety  Market 
Knowledge:  How  the 
healthcare  institution 
determines  patients’ 
expectations  and  appropriate 
knowledge  in  regard  to  patient 
safety. 

Satisfaction  Determination: 
How  the  institution  obtains 
information  and  feedback  from 
patients  on  patient  safety 
issues  to  improve  the  delivery 
of  healthcare. 


Planned,  coordinated,  and  aligned 
institutional  activities  to  ensure  patient 
education  and  providing  of  useful 
information  to  the  intended  audiences 
in  regard  to  patient  safety  issues, 
institutional  policies,  and  practices. 

Design  and  implementation  of 
comprehensive  and  accessible 
systems  for  adverse  events  reporting 
from  patients  and  their  families,  and 
continuous  analysis  of  the  obtained 
data. 


Category  4:  Measurement,  Analysis,  and  Knowledge  Management 


4.1.  Measurement  and  Performance  Measurement: 
Analysis  of  Institutional  How  the  institution  collects, 
Performance  tracks,  and  analyzes  patient 

safety  data. 


4.2.  Information  and 

Knowledge 

Management 


Data  and  Information 
Availability:  How  the  institution 
ensures  that  its  clinical 
information  technology 
(Computerized  Physician's 
Order  Entry  -  CPOE,  infusion 
pumps,  alarm  systems,  etc.)  is 
reliable,  secure,  and  user- 
friendly. 


Nonpunitive  reporting  systems  are  in 
place  for  recording,  monitoring, 
tracking,  and  analysis  of  adverse 
events  and  near  misses  and  the 
results  from  this  analysis  are  used  in 
institution’s  improvement  plans. 

A  planned,  aligned,  and  monitored 
institution-wide  process  of  clinical 
technology  use  facilitates  information 
transfer  and  clear  communication. 
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Table  2.  Patient  safety  applicator  tool:  beginners,  cont. 

Item  Area  to  address:  Critical  Performance  Measure 

Process 

Category  5:  Staff  Focus 

5.1.  Work  Systems  Staff  Performance  Management  Best  patient  safety  practices  and 

System:  How  the  institution  clinical  guidelines  are  adopted  and 

supports  high  clinical  monitored,  and  clinician  performance 

performance  standards  and  is  evaluated  for  consistency  with  these 

alignment  with  national  clinical  adopted  standards. 

performance  measures  and 

best  case-management 

practices. 

5.2.  Staff  Learning  and  Staff  Education.  Training,  and  Institutional  mechanism  for 

Motivation  Development:  How  the  determining  of  and  acting  on  patient 

institution  structures  and  safety  educational  and  training  needs 

promotes  effective  education  for  individuals,  teams,  departments, 
and  training  of  professionals  in  and  different  categories  of 
developing  and  improving  professional  caregivers, 

patient  safety  systems. 

5.3.  Staff  Well-being  and  Work  Environment:  How  the  Institution’s  patient  safety  goals  are 

Satisfaction  institution  maintains  conducive  integrated  in  institution’s  everyday 

environment  in  regard  to  patient  healthcare  delivery  functions,  regularly 
safety.  reviewed  and  improved,  and  progress 

towards  them  is  continuously 
monitored  and  evaluated. 

Category  6:  Process  Management 

Patient  Safety  System:  How  the  An  institutional  mechanism  exists  for 
institution  ensures  that  patient  continuous  monitoring,  improvement, 
safety  requirements  are  met  at  and  sustainability  of  patient  safety 
the  “sharp  end”  of  the  outcomes  in  healthcare  delivery, 

healthcare  delivery  system. 

Patient  Safety  Support  Systems  for  departmental  and 

Processes:  How  the  institution  interdepartmental  communications, 
coordinates  departmental  and  collaborations,  and  aligned  effort  in 
interdepartmental  patient  safety  regard  to  seamless  implementation  of 
infrastructures  to  reduce  best  practices  and  clinical  guidelines 

variability  in  healthcare  delivery  in  patient  identification,  medication, 
and  improve  performance.  and  continuous  case  management  are 

assessed  and  improved  on  an  ongoing 
basis. 

Category  7:  Institutional  Performance 

7.1.  Patient  Safety  Patient  Safety  Results:  How  the  The  institution  monitors  the 

Institutional  institution  ensures  patient  administration  of  high-alert 

Performance  safety.  medications. 


Summary  of  findings 

This  study  identified  a  systems-based  patient  safety  applicator  tool  useful  for 
assessing  patient  safety  systems  in  health  care  institutions.  The  advanced  level  of 
the  tool  has  39  critical  processes  and  60  perfonnance  measures.  The  Delphi  panel 
consensus  showed  that,  at  present,  leadership  involvement  and  direction  is  the 
major  factor  for  building,  maintaining,  and  improving  patient  safety  systems.  The 
Delphi  panel  also  forecasted  that  patient  safety  measurement,  analysis,  and 
knowledge  management  will  be  the  leading  factors  in  improving  patient  safety 


6.1.  Patient  Safety 
System 


6.2.  Support  Processes 
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systems  in  the  future.  In  addition,  the  study  results  supported  the  findings 
identified  in  the  literature  review  that  more  aggressive  use  of  quality 
measurement,  data  analysis,  and  knowledge  management  is  needed  in  health  care 
settings. 

The  Delphi  panel  reached  consensus  that: 

•  Utilization  of  national  patient  safety  benchmarks  and  clinical  protocols 
is  an  important  aspect  of  patient  safety. 

•  Standardization  of  health  care  devices,  treatment  approaches,  and 
services  is  necessary  in  achieving  patient  safety. 

•  Technology  will  be  one  of  the  leading  change  agents  for  improving 
patient  safety  in  health  care  institutions.  Patient  safety  approaches, 
such  as  utilization  of  a  computerized  physician’s  order  entry,  patient 
barcodes,  computerized  phannacology  dispensing  systems,  and 
reliable  infusion  pumps  and  alarm  systems,  have  potential  for 
decreasing  medical  errors  and  making  health  care  safer. 

•  Patient  safety  education  of  health  care  leadership,  clinical  staff,  and 
patients  is  critical  for  improving  patient  safety. 

At  present,  the  prioritization  of  the  patient  safety  applicator  tool  categories 
based  on  the  consensus  group  rank  mean  of  all  processes  within  each  category  is 
as  follows: 

1.  Category  1,  Leadership. 

2.  Category  4,  Measurement,  analysis,  and  knowledge  management. 

3.  Category  2,  Strategic  planning,  and  Category  6,  Process  management. 

4.  Category  5,  Staff  focus. 

5.  Category  3,  Focus  on  patients,  other  customers,  and  markets. 

This  hierarchy  identified  by  the  experts  within  the  interrelated  MBNQA 
categories  shows  that  at  present  the  commitment  to  patient  safety  by  institution’s 
leadership  is  undoubtedly  the  moving  force  for  institution-wide  adoption  of 
patient  safety  systems.  Patient  safety  outcomes  measurement,  at  the  institutional 
and  provider  levels,  has  gained  popularity  and  recognition  with  recently  mandated 
patient  safety  outcome  requirements.  This  hierarchy  illustrates  that  the  way  to 
start  a  systemwide  patient  safety  improvement  is  to  ensure  leadership  knowledge, 
understanding,  and  commitment  to  patient  safety. 

The  importance  of  the  patient  safety  applicator  tool  categories  in  the  future,  as 
forecasted  by  the  Delphi  experts,  is  as  follows: 

1 .  Category  4,  Measurement,  analysis,  and  knowledge  management. 

2.  Category  1,  Leadership;  and  Category  2,  Strategic  planning. 

3.  Category  5,  Staff  focus. 

4.  Category  6,  Process  management. 
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5.  Category  3,  Focus  on  patients,  other  customers,  and  markets. 

The  experts’  forecast  for  the  future  recognizes  the  growing  importance  of  and 
emphasis  on  patient  safety  outcome  data  as  the  driving  force  for  further 
establishment  and  strengthening  of  the  business  case  for  patient  safety. 
Measurement  and  analysis  of  patient  safety  outcome  data,  along  with 
management  of  patient  safety  knowledge,  will  support  institutional  leadership  in 
choosing  the  direction  for  the  next  patient  safety  steps  in  individual  health  care 
organizations. 

The  low  priority  given  to  Category  3  (Focus  on  patients,  other  customers,  and 
markets),  both  in  the  present  and  in  the  future,  shows  that  health  care 
administrators  and  professionals  do  not  fully  appreciate  the  importance  of 
addressing  health  care  customer  and  market  requirements.  It  is  obvious  that  health 
care  administrators  and  professionals  are  still  uncomfortable  focusing  on 
customers  and  markets.  The  results  for  performance  measures  in  Category  3  are  in 
sync  with  the  traditional  service  model  in  health  care.  Health  care  professionals 
have  been  accustomed  to  a  service  model  where  the  customer  has  been  narrowly 
defined  to  include  patients  and  their  families,  or  even  more  narrowly  defined  as 
including  only  the  patients.  In  this  old  service  model  the  customers  have  been 
seen  as  “recipients”  of  care  rather  than  as  active  participants  in  the  process  of 
care.  In  the  new  marketplace,  such  a  service  model  is  not  only  unsuitable,  but  also 
inappropriate  if  the  health  care  institutions  want  to  re-focus  on  quality  and  safety 
of  care,  providing  patient-centered  services,  and  building  a  strong  business  case 
for  patient  safety.  Similar  to  borrowing  performance  measurement  approaches 
from  other  industries  (e.g.,  aviation,  automotive  industry,  etc.),  borrowing 
customer  and  market  approaches  from  other  industries  (e.g.,  from  other  service 
industries)  may  prove  beneficial  for  focusing  health  care  institutions’  attention  on 
better  serving  their  customers  and  expanding  their  market  share. 

The  outcomes  of  this  study  suggest  that  to  implement,  maintain,  and  improve 
patient  safety  systems,  health  care  administrators  should  do  the  following: 

•  Effectively  communicate  the  priority  of  patient  safety  to  all 
stakeholders,  actively  seeking  feedback  on  patient  safety  and  using  the 
information  for  patient  safety  improvements. 

•  Ensure  ethical  communication  with  stakeholders  in  regard  to  patient 
safety  and  utilize  ongoing  monitoring  and  analysis  of  patient  safety 
outcomes  for  patient  safety  improvement. 

•  Incorporate  national  databank  data  and  clinical  practice  guidelines  as 
patient  safety  performance  benchmarks  in  institution’s  strategic  plan 
and  monitor  staff  performance  against  the  adopted  benchmarks. 

•  Develop,  monitor,  regularly  review,  and  improve  institution’s  patient 
safety  action  plans. 

•  Plan,  coordinate,  and  align  institutional  patient  safety  activities  to 
ensure  patient  safety  education  of  institution’s  leadership,  medical 
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staff,  and  patients  at  the  individual,  team,  departmental,  and 
institutional  levels,  as  appropriate. 

•  Design  and  implement  comprehensive,  accessible,  and  user-friendly 
systems  for  recording,  monitoring,  tracking,  and  analyzing  adverse 
events  and  near  miss  reporting  from  both  staff  and  patients,  and  use 
the  results  of  the  analysis  for  further  patient  safety  improvement. 

•  Plan,  align,  and  monitor  institution-wide  processes  for  facilitation  of 
information  transfer  and  communications  through  clinical  information 
technology. 

•  Integrate  institution’s  patient  safety  goals  into  everyday  health  care 
delivery  functions  and  protocols,  regularly  review  and  improve  the 
health  care  delivery  protocols,  and  continuously  monitor  and  evaluate 
institution’s  performance  toward  its  safety  goals. 

•  Adopt  clinical  protocols  for  administration  of  high-alert  medications. 

•  Coordinate  departmental  and  interdepartmental  patient  safety 
infrastructures  to  reduce  variability  in  health  care  delivery  and  to 
improve  performance. 

•  Design  and  implement  an  institutional  mechanism  for  continuous 
monitoring,  improvement,  and  sustainability  of  patient  safety 
outcomes  in  health  care  delivery. 

The  experts  on  the  Delphi  panel  emphasized  the  importance  of  the  “non- 
punitive”  aspect  of  medical  error  reporting  systems  based  on  the  specific  message 
that  health  care  professionals  should  not  feel  threatened  when  reporting  medical 
errors  or  near  misses.  Designing  and  utilizing  nonpunitive  and  reliable  reporting 
systems  that  allow  confidential  or  anonymous  reporting  of  adverse  events  and 
near  misses  is  essential  if  health  care  institutions  are  to  become  learning 
organizations  and  if  they  are  to  use  medical  error  and  near  miss  data  for 
improvement  of  their  patient  safety  systems. 


Discussion 

The  Institute  of  Medicine  (IOM)  has  recognized  the  underdevelopment  of 
patient  safety  systems  in  most  U.S.  health  care  settings  and  has  rigorously 
emphasized  the  need  for  implementation  of  comprehensive  patient  safety  systems 
as  an  integral  part  of  all  health  care  organizations.17  Patient  safety  systems,  which 
are  multi-level  and  multifaceted,  include  a  culture  of  safety  and  strong 
organizational  support  for  implementation  and  improvement. 

Physician  clinical  perfonnance  measures  intend  to  measure  individual 
physician’s  clinical  practice  behavior  and  adherence  to  evidence-based,  objective 
clinical  practice  criteria.  Thus,  such  an  assessment  is  based  on  the  availability  of 
clinical  evidence  that  certain  processes  and  behaviors  can  be  linked  to  patient 

1  Q 

outcomes.  Such  evidence  is  limited  for  the  majority  of  clinical  specialties. 
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Therefore,  it  is  very  difficult  to  formulate  robust  clinical  performance  measures, 
and  considerable  variation  will  continue  to  exist  between  clinical  specialties  and 
facilities.  In  order  to  expand  the  relationship  between  its  accreditation  criteria  and 
the  MBNQA  framework,  as  well  as  to  ensure  compatibility  with  the  principles  of 
quality  management,  the  Joint  Commission  on  Accreditation  of  Healthcare 
Organizations  (JCAHO)  re-modeled  its  accreditation  process  and  introduced 
Tracer  Methodology  (part  of  the  Shared  Visions  -  New  Pathways  accreditation 
process),  which  is  focused  on  processes  critical  to  patient  safety.19,20 

Traditionally,  efforts  toward  improvement  in  medicine  have  more  often  than 
not  targeted  individuals  and  centered  on  training,  rules,  and  sanctions  rather  than 
on  systems  and  system  failures.2 1-25  However,  medical  errors  occur  as  the  result 
of  a  chain  of  errors  within  a  faulty  system  that  is  not  designed  to  detect  errors  and 
intercept  them.  Thus,  while  individual  responsibility  for  deviating  from  policies 
and  procedures  remains  important,  errors  can  be  eliminated  only  through  focusing 
beyond  the  individual  and  focusing  on  systems  design  changes  supported  by  top- 
level  management.17,24, 26  29 

Patient  safety  and  quality  definitions  continue  to  be  the  focus  of  expert 
debates.  For  the  last  several  years  the  Agency  for  Healthcare  Research  and 
Quality  (AHRQ)  has  sponsored  patient  safety  research  to  identify  patient  safety 
risks,  design  patient  safety  practices,  educate  health  care  professionals,  and 
monitor  patient  safety  trends.  It  also  has  started  initiatives  to  identify  best  clinical 
practices  and  train  patient  safety  researchers.  Other  agencies,  such  as  the  Centers 
for  Disease  Control  and  Prevention,  Food  and  Drug  Administration,  and  Veterans 
Administration,  have  supported  patient  safety  research  and  patient  safety  activities 
as  well  as  legislative  efforts  focused  on  creating  voluntary  medical-errors 
reporting  systems.  Despite  these  efforts,  patient  safety  improvements  will  not  be 
achieved  without  standardizing  patient  safety  definitions,  terminology, 
measurements,  and  databases.17,30  Essential  steps  include  utilizing  quality 
improvement  tools,  recognizing  the  systemic  roots  of  medical  errors,  performing 
failure  mode  and  effects  analysis  (FMEA),  designing  fail-safe  systems,  and 
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training  medical  professionals  in  error  reduction  team  processes.  ’  ’  ’ 

Patient  safety  is  a  subset  of  health  care  quality.  Therefore,  successful  quality 
improvement  programs  should  start  with  setting  the  scene  (i.e.,  institution-wide 
policies  and  procedures  for  deploying  systems  for  clinical  performance 
assessment  as  well  as  recognizing,  analyzing,  and  dealing  with  the  systems  nature 
of  medical  errors).  ’  "  Health  care  processes  are  interdepartmental  in  nature  and 
should  be  designed  and  assessed  by  cross-functional  teams,  including  clinical, 
administrative,  and  other  staff  members.  Thus,  linking  patient  information 
systems  to  administrative  and  financial  information  systems  appears  critical  for 
measuring  health  care  outcomes.17, 19  The  role  of  leadership  in  using  process  data 
for  improvement  rather  than  for  disciplinary  actions  is  extremely  important  in  the 
patient  safety  improvement  effort.  Performance  data  collection  should  be  based 
on  accepted  performance  guidelines  sets,  such  as  accreditation  standards  or  the 
Baldrige  Healthcare  Criteria  for  Performance  Excellence. 
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Health  care  futurists  have  put  the  issues  of  patient  safety  and  quality 
improvement,  along  with  the  issues  of  implementation  of  clinical  information 
systems,  e-health,  and  electronic  medical  records,  among  the  top  trends  in  U.S. 

'xn 

health  care.  The  problem  in  health  care  is  not  the  lack  of  tools  to  improve  patient 
safety,  but  rather  the  low  priority  traditionally  assigned  to  this  goal.38  While 
quality  experts  agree  that  the  cause  for  the  majority  of  performance  problems  is 
the  health  care  delivery  system  itself,  and  accident  investigators  agree  that  most 
disasters  in  complex  organizations  have  long  incubation  periods  and  manifest 
with  multiple  discrete  “small”  events  over  a  long  period  of  time,  the  study  of 
human  error  in  medicine  is  a  relatively  new  field  that  is  trying  to  establish  its 
boundaries,  terminology,  and  taxonomy.  Additionally,  the  current  focus  of 
attention  is  still  predominantly  on  individual  perfonnance  and  responsibility. 

Since  health  care  organizations  usually  create  department  or  service-specific 
measures  and  reports,  data  are  collected  and  reported  separately,  and  no  particular 
individual  or  department  has  comprehensive  information  on  the  whole 
organizational  picture.39 

Reason40  developed  a  model  for  organizational  accidents  in  complex 
industrial  systems  that  considered  not  only  the  actions  of  the  individuals  involved, 
but  also  the  conditions  in  which  the  tasks  were  performed  and  the  organizational 
context  in  which  the  incident  occurred.  He  distinguished  between  active  failures 
as  unsafe  acts  of  omission  or  commission  and  latent  failures  stemming  from 
managerial  decisions,  process  organization,  and  system  design.  Thus,  latent 
failures  provide  the  work  conditions  where  unsafe  acts  occur.  Such  conditions 
include  inadequate  knowledge,  training  or  expertise,  heavy  workloads,  inadequate 
supervision,  inadequate  systems  of  communication,  inadequate  maintenance  of 
plant  and  equipment,  and  stressful  environment.40  The  accident  opportunity  has  to 
penetrate  through  several  layers  of  defense  systems  on  managerial,  psychological, 
environmental,  and  local  levels  before  an  opportunity  window  (also  called  “the 
Swiss  cheese  model”)  allows  the  error  to  reach  a  patient.40'41  Thus,  discipline 
should  be  not  the  first,  but  rather  the  last  action  taken  against  staff  members  as  a 
result  of  reported  errors.  Focusing  on  systems  analysis,  education,  development, 
and  dissemination  of  clinical  practice  guidelines;  using  automated  “fail-safe” 
systems;  and  utilizing  computer  reminders  have  been  identified  as  prevention 
factors  for  adverse  events  in  health  care.  Studying  these  factors  and  their 
interrelations  has  been  recommended  for  prevention  of  medical  errors.43  Patient 
safety,  as  a  property  of  the  health  care  systems,  can  be  created  or  broken  in 
systems,  and  correcting  systems’  vulnerabilities  is  the  efficient  way  to  go.44 

Successful  health  care  leaders  for  the  21st  century  are  expected  to  build  the 
capacity  of  the  health  care  system  for  improvement.  Chief  executive  officers  in 
health  care  organizations  should  be  able  to  recognize  the  importance  of  aligning 
all  organizational  strategies  for  improvement — operations,  professional 
development,  and  financing — and  systematically  approach  organizational  and 
personal  improvement  through  performing  patient  assessment,  process 
assessment,  data  gathering,  and  critical  assessment  of  the  current  literature.17'43  It 
is  widely  agreed  that  systems  lie  at  the  base  of  the  majority  of  medical  errors  and 
that  improvement  can  be  achieved  through  system  re-design  and  institutional  self- 


121 


Advances  in  Patient  Safety:  Vol.  4 


assessment.17' 30' 34  The  responsibility  for  creating  and  endorsing  such  an 
institution-wide  improvement  system  ultimately  resides  within  the  health  care 
executive  leadership.  A  system  for  patient  safety  improvement  should  be  based  on 
the  principles  of  simplification,  standardization,  stratification,  improved 
communication,  designing  easy  “default”  procedures,  reasonable  automation, 
process  mapping,  recognition  of  the  limitations  of  human  vigilance,  and 
encouragement  of  error  reporting.  As  patient  safety  was  declared  an  issue  of 
major  national  interest  with  the  series  of  IOM  publications,17’ 46' 47  policymakers, 
health  care  professionals,  and  consumers  started  recognizing  the  need  to  assess, 
monitor,  and  improve  health  care  safety. 

The  interrelationship  and  interdependence  of  the  MBNQA  categories  illustrate 
that  it  is  important  for  institutions  to  develop  all  MBNQA  categories.  The  patient 
safety  tool  resulting  from  this  study  utilizes  systems  approach  for  advancing  the 
MBNQA  model  in  the  area  of  patient  safety.  Within  the  MBNQA  framework,  the 
study  participants  identified  a  hierarchy  of  categories,  both  for  the  present  and  the 
future,  that  will  assist  health  care  administrators  in  selecting  the  category  needing 
most  improvement  in  their  respective  institutions.  It  is  important  to  start  with 
leadership  commitment  to  lead  the  patient  safety  process  at  the  institutional  level. 
In  the  future,  measurement  and  analysis  of  patient  safety  data  will  continue  to 
drive  leadership  direction,  funds  allocation,  and  strategic  planning  activities. 
Patient  safety  problems  occur  frequently  and  with  significant  impact  on  individual 
health  care  facilities  and  the  health  care  industry  as  a  whole.  A  strong  business 
case  for  patient  safety  and  institution-wide  support  for  implementation, 
management,  and  improvement  of  patient  safety  systems  will  justify  allocation  of 
appropriate  funds  and  required  process  changes.17'27, 30, 46, 47  The  MBNQA  criteria 
give  a  structured  matrix  for  producing  results  and  can  be  used  as  an  objective  tool 
for  building  the  business  case  for  patient  safety.  The  applicator  tool  based  on  the 
MBNQA  criteria  provides  a  venue  for  health  care  organizations  to  study  patient 
safety  issues  within  an  organization  and  to  understand  which  components  need  to 
work  together  to  accelerate  patient  safety  outcome  improvement. 


Conclusions 

Patient  safety  practices  are  multidimensional  and  difficult  to  assess  and  reach 
through  all  organizational  levels.  Implementation  of  patient  safety  systems  based 
on  the  MBNQA  model  may  ensure  that  health  care  institutions  provide  the 
systemic  approach  to  quality  services  they  intend  to  provide.  The  critical 
processes  and  performance  measures  identified  in  this  research  will  be  useful  for 
health  care  institutions  in  designing,  implementing,  and  improving  patient  safety 
systems.  This  study  used  the  MBNQA  model,  a  proven  model  that  has  been 
refined  over  the  past  17  years  and  that  has  been  adopted  nationally  by  numerous 
health  care  organizations  to  manage  and  advance  improvement  strategies  in  their 
systems.  The  approach  to  patient  safety  thus  far  has  been  reflective  primarily  of 
departmental  issues,  or  has  been  led  by  narrowly  tailored,  disease-based  priorities 
and  outcomes.  According  to  the  current  patient  safety  approach,  specific  patient 
outcomes,  processes,  or  activities  that  have  not  been  measured  are  not  reflected, 
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analyzed,  or  taken  into  consideration  for  future  patient  safety  initiatives.  In  light 
of  this,  a  systems  approach,  based  on  the  already  refined  Malcolm  Baldrige 
model,  offers  the  advantage  of  a  more  holistic  approach  to  building  patient  safety 
systems  at  the  institutional  level.  The  patient  safety  applicator  tool,  which  extends 
the  Malcolm  Baldrige  systems  model  to  patient  safety,  is  intended  to  support 
senior  health  care  administrators  in  achieving  and  sustaining  improvement  results 
and  may  also  serve  as  a  means  for  evaluating  existing  patient  safety  initiatives  or 
for  guiding  the  planning  of  new  processes  for  better  health  care  delivery. 
Moreover,  the  identified  patient  safety  applicator  tool  will  assist  health  care 
institutions  in  using  the  MBNQA  Health  Care  Criteria  for  Performance 
Excellence  for  self-assessment  and  quality  improvement.  The  three  levels  of  this 
patient  safety  applicator  tool  are  intended  to  meet  the  needs  of  health  care 
organizations  that  are  at  different  stages  of  their  patient  safety  journey. 
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